Deployment: Model-agnostic methods
library(randomForest)
library(dplyr)
library(mltools)
library(data.table)
library(pdp)
library(plotly)
horas <- read.csv('hour.csv')
datos <- read.csv('day.csv')
house <- read.csv('kc_house_data.csv')
1.- One dimensional Partial Dependence Plot.
The partial dependence plot shows the marginal effect of a feature on
the predicted outcome of a previously fit model.
Apply PDP to the regression example of predicting bike rentals. Fit a
random forest approximation for the prediction of bike rentals
(cnt). Use the partial dependence plot to visualize the
relationships the model learned. Use the slides shown in class as
model.
#Cargamos los datos
days$dteday <- as_date(days$dteday) # pasamos de caracter a fecha
#Selección de variables para el modelo:
datmod <- select(days, workingday, holiday, temp, hum, windspeed, cnt)
datmod$days_since_2011 <- int_length(interval(ymd("2011-01-01"), days$dteday)) / (3600*24)
#Creamos las nuevas variables para el modelo:
datmod$winter <- ifelse(days$season==1, 1, 0)
datmod$summer <- ifelse(days$season==3, 1, 0)
datmod$fall <- ifelse(days$season==4, 1, 0)
datmod$MISTY <- ifelse(days$weathersit == 2, 1, 0)
datmod$RAIN <- ifelse(days$weathersit == 3 | days$weathersit == 4, 1, 0)
#Al normalizar las variables se hace dificil interpretar el modelo, ya que los valores normalizados no tienen una interpretacion intuitiva. Al desnormalizar las variables nos permite interpretar los coeficientes del modelo individualmente de un modo mas sencillo. Pero, perdemos la capacidad de comparar las varibales, ya que cada uno tiene su propio rango de valores y escalas originales.
#t_norm * (t_max-t_min) + t_min = t
datmod$temp <- days$temp * (39-(-8)) - 8
#hum_nor = hum/100
datmod$hum <- days$hum * 100
#wind_nor = wind/67
datmod$windspeed <- days$windspeed * 67
library(pdp)
library(vip)
#Entrenamos el modelo
days_rf <- randomForest(cnt ~ ., data = datmod, importance = TRUE)
p1 <- partial(days_rf, pred.var = "days_since_2011", plot = TRUE, rug = TRUE, plot.engine = "ggplot2")
p2 <- partial(days_rf, pred.var = "temp", plot = TRUE, rug = TRUE, plot.engine = "ggplot2")
p3 <- partial(days_rf, pred.var = "hum", plot = TRUE, rug = TRUE, plot.engine = "ggplot2")
p4 <- partial(days_rf, pred.var = "windspeed", plot = TRUE, rug = TRUE, plot.engine = "ggplot2")
subplot(p1,p2,p3,p4, shareX = F, titleX = TRUE)
Analyse the influence of days since 2011,
temperature, humidity and wind
speed on the predicted bike counts.
As time goes on, the model predicts an increase in the number of
rented bicycles, which is normal as the service becomes more well-known
over time. For warm but not too hot climates, a large number of rented
bikes is predicted. Yet, from temperatures over 27 ºC, the number of
rented bikes decrease(too much heat). It appears that cyclists are
increasingly inhibited from renting a bike when humidity exceeds 60%.
Finally, the windier it gets, the less people like to ride a bike. which
is logic. It appears that the model predicts the same from 25 km/h,
maybe because there is little training data in that range.
2.- Bidimensional Partial Dependency Plot.
Generate a 2D Partial Dependency Plot with humidity and temperature
to predict the number of bikes rented depending on those parameters.
Show the density distribution of both input features with the 2D plot
as shown in the class slides.
library(tictoc)
#Seleccionamos un número de filas aleatorias
sampled <- sample_n(datmod, 40)
#Filas del atributo 'temp' que tienen una equivalencia en el atributo 'hum'
temphum <- inner_join(data.frame(sampled$temp), data.frame(sampled$hum), by=character())
colnames(temphum) <- c("temperature","humidity")
temphum$prob <- 0
for(i in 1:nrow(temphum)){
r <- datmod
r[["temp"]] <- temphum[["temperature"]][i]
r[["hum"]] <- temphum[["humidity"]][i]
pred <- predict(days_rf, r)
temphum[["prob"]][i] <- sum(pred) / nrow(datmod)
}
ggplot(temphum, aes(x=temperature, y=humidity)) + geom_tile(aes(fill=prob, width=10, height=10)) + labs(x="Temperature", y="Humidity") + guides(fill=guide_legend(title="Number of bikes")) + geom_rug()

QUESTION:
For the analysis of the two-dimensional PDP we must consider in
unison the density graphs of each of the attributes (represented on
their respective axes) with the legend which, by the intensity of the
colour, indicates the estimated value of the number of bicycles rented.
Thus, this legend shows that the areas with a lighter blue colour
indicate a higher value of the predicted response variable, and
alternatively the darker tones correspond to lower estimated values for
the number of bicycles.
We focus our explanation on the ranges of values of our attributes
for which we can draw general interpretations with respect to the
predictive model. In our case for temperature (from 4ºC to 28ºC) and for
humidity (from 44% to 81%) we observe nine clearly differentiated zones
in terms of tone.
In the first zone (Temperature: 4-8ºC, Humidity: 75-81%) we observe
that it is where the number of rented bicycles is the lowest (between
3000 and 3500 units) just where low temperature and high humidity
conditions meet.
In the last zone (temperature: 17-28ºC, humidity: 44-61%) delimited
by high temperature and low humidity conditions, the highest number of
rented bicycles is predicted (approximately 5000 units).
Both boundary situations correspond to the direct relationship
between our response variable with the variable ‘temp’ as well as the
inverse relationship with the variable ‘hum’.
3.- PDP to explain the price of a house.
Apply the previous concepts to predict the price of
a house from the database kc_house_data.csv. In this
case, use again a random forest approximation for the prediction based
on the features bedrooms, bathrooms,
sqft_living, sqft_lot,
floors and yr_built.
Use the partial dependence plot to visualize the relationships the
model learned.
#Selección de filas aleatorias
sample_house <- sample_frac(house, 0.2)
sample_house <- select(sample_house, bedrooms, bathrooms, sqft_living, sqft_lot, floors, yr_built, price)
#Entrenamos el modelo
house_rf <- randomForest(price ~ ., data = sample_house, importance = TRUE)
p1 <- partial(house_rf, pred.var = "bedrooms", plot = TRUE, rug = TRUE, plot.engine = "ggplot2")
p2 <- partial(house_rf, pred.var = "bathrooms", plot = TRUE, rug = TRUE, plot.engine = "ggplot2")
p3 <- partial(house_rf, pred.var = "sqft_living", plot = TRUE, rug = TRUE, plot.engine = "ggplot2")
p4 <- partial(house_rf, pred.var = "floors", plot = TRUE, rug = TRUE, plot.engine = "ggplot2")
subplot(p1,p2,p3,p4, shareX = FALSE, titleX = TRUE)
NA
QUESTION:
Analyse the influence of bedrooms, bathrooms, sqft_living
and floors on the predicted price.
The general interpretation of the ‘post hoc’ PDP model for the
attribute ‘bedrooms’ with respect to the predicted
model according to its density graph allows us to draw conclusions in
the range of 1 to 4 bedrooms. We observe two different trends: an
increasing trend, i.e. a rise in price from 1 to 2 bedrooms (from 572000
to 581000\(); while a decrease in price from 2
to 4 bedrooms (from 581000 to 542000\)). We can understand that
the most requested number of rooms per house, and therefore the most
expensive, are those with 2 rooms. In the range above these, as it is
not a usual number of rooms, we interpret this as the reason why the
predictive model justifies the trend described above.
As for the variable ‘bathrooms’ we can observe a
positive correlation between it and the estimated price of the house,
since normally a house with more bathrooms is considered a more
luxurious house and therefore more expensive, it is worth noting the
clear increase in price if we go from 3 to 4 bathrooms. We can only draw
valid conclusions if we take into account those dwellings with 0 to 4
bathrooms, as we do not have enough data recorded for dwellings with a
higher number of bathrooms.
As for the variable ‘sqft_living’, the relationship
is similar to the case of bathrooms, a larger dwelling will obviously be
more expensive as it occupies more land. We have data up to about 5000
square metres, for those dwellings whose size exceeds this, we do not
have enough data recorded and therefore cannot draw valid
conclusions.
For the last PDP graph, in relation to the analysis of the variable
‘floors’ being the increasing trend we observe a
notable difference in the price of the house when going from one floor
to two, and above all, if there is a third floor in the house.
Considering the small difference between one floor and two floors (from
540000 to 560000$), taking into account the demographic characteristics
mentioned above and considering the climatological characteristics of
the area (short, hot and dry summers and cold and wet winters) the
construction estimated by the two-floor model is usually the most
demanded, placing the living areas and kitchen on one floor and the
bedrooms on the upper floor. Of course, a third floor is already a big
budget deviation and is considered to be an exceptional house.
LS0tDQp0aXRsZTogIlBSNSBERVBMT1lNRU5UIg0Kb3V0cHV0OiBodG1sX25vdGVib29rDQotLS0NCg0KIyBEZXBsb3ltZW50OiBNb2RlbC1hZ25vc3RpYyBtZXRob2RzDQoNCmBgYHtyfQ0KbGlicmFyeShyYW5kb21Gb3Jlc3QpDQpsaWJyYXJ5KGRwbHlyKQ0KbGlicmFyeShtbHRvb2xzKQ0KbGlicmFyeShkYXRhLnRhYmxlKQ0KbGlicmFyeShwZHApDQpsaWJyYXJ5KHBsb3RseSkNCmBgYA0KDQpgYGB7cn0NCmhvcmFzIDwtIHJlYWQuY3N2KCdob3VyLmNzdicpDQpkYXRvcyA8LSByZWFkLmNzdignZGF5LmNzdicpDQpob3VzZSA8LSByZWFkLmNzdigna2NfaG91c2VfZGF0YS5jc3YnKQ0KYGBgDQoNCiMjIyAqKjEuLSBPbmUgZGltZW5zaW9uYWwgUGFydGlhbCBEZXBlbmRlbmNlIFBsb3QuKioNCg0KVGhlIHBhcnRpYWwgZGVwZW5kZW5jZSBwbG90IHNob3dzIHRoZSBtYXJnaW5hbCBlZmZlY3Qgb2YgYSBmZWF0dXJlIG9uIHRoZSBwcmVkaWN0ZWQgb3V0Y29tZSBvZiBhIHByZXZpb3VzbHkgZml0IG1vZGVsLg0KDQpBcHBseSBQRFAgdG8gdGhlIHJlZ3Jlc3Npb24gZXhhbXBsZSBvZiBwcmVkaWN0aW5nIGJpa2UgcmVudGFscy4gRml0IGEgcmFuZG9tIGZvcmVzdCBhcHByb3hpbWF0aW9uIGZvciB0aGUgcHJlZGljdGlvbiBvZiBiaWtlIHJlbnRhbHMgKCoqY250KiopLiBVc2UgdGhlIHBhcnRpYWwgZGVwZW5kZW5jZSBwbG90IHRvIHZpc3VhbGl6ZSB0aGUgcmVsYXRpb25zaGlwcyB0aGUgbW9kZWwgbGVhcm5lZC4gVXNlIHRoZSBzbGlkZXMgc2hvd24gaW4gY2xhc3MgYXMgbW9kZWwuDQoNCmBgYHtyfQ0KI0NhcmdhbW9zIGxvcyBkYXRvcyANCmRheXMkZHRlZGF5IDwtIGFzX2RhdGUoZGF5cyRkdGVkYXkpICMgcGFzYW1vcyBkZSBjYXJhY3RlciBhIGZlY2hhIA0KDQoNCiNTZWxlY2Npw7NuIGRlIHZhcmlhYmxlcyBwYXJhIGVsIG1vZGVsbzoNCmRhdG1vZCA8LSBzZWxlY3QoZGF5cywgd29ya2luZ2RheSwgaG9saWRheSwgdGVtcCwgaHVtLCB3aW5kc3BlZWQsIGNudCkNCmRhdG1vZCRkYXlzX3NpbmNlXzIwMTEgPC0gaW50X2xlbmd0aChpbnRlcnZhbCh5bWQoIjIwMTEtMDEtMDEiKSwgZGF5cyRkdGVkYXkpKSAvICgzNjAwKjI0KQ0KDQojQ3JlYW1vcyBsYXMgbnVldmFzIHZhcmlhYmxlcyBwYXJhIGVsIG1vZGVsbzoNCmRhdG1vZCR3aW50ZXIgPC0gaWZlbHNlKGRheXMkc2Vhc29uPT0xLCAxLCAwKQ0KZGF0bW9kJHN1bW1lciA8LSBpZmVsc2UoZGF5cyRzZWFzb249PTMsIDEsIDApDQpkYXRtb2QkZmFsbCA8LSBpZmVsc2UoZGF5cyRzZWFzb249PTQsIDEsIDApDQpkYXRtb2QkTUlTVFkgPC0gaWZlbHNlKGRheXMkd2VhdGhlcnNpdCA9PSAyLCAxLCAwKQ0KZGF0bW9kJFJBSU4gPC0gaWZlbHNlKGRheXMkd2VhdGhlcnNpdCA9PSAzIHwgZGF5cyR3ZWF0aGVyc2l0ID09IDQsIDEsIDApDQoNCiNBbCBub3JtYWxpemFyIGxhcyB2YXJpYWJsZXMgc2UgaGFjZSBkaWZpY2lsIGludGVycHJldGFyIGVsIG1vZGVsbywgeWEgcXVlIGxvcyB2YWxvcmVzIG5vcm1hbGl6YWRvcyBubyB0aWVuZW4gdW5hIGludGVycHJldGFjaW9uIGludHVpdGl2YS4gQWwgZGVzbm9ybWFsaXphciBsYXMgdmFyaWFibGVzIG5vcyBwZXJtaXRlIGludGVycHJldGFyIGxvcyBjb2VmaWNpZW50ZXMgZGVsIG1vZGVsbyBpbmRpdmlkdWFsbWVudGUgZGUgdW4gbW9kbyBtYXMgc2VuY2lsbG8uIFBlcm8sIHBlcmRlbW9zIGxhIGNhcGFjaWRhZCBkZSBjb21wYXJhciBsYXMgdmFyaWJhbGVzLCB5YSBxdWUgY2FkYSB1bm8gdGllbmUgc3UgcHJvcGlvIHJhbmdvIGRlIHZhbG9yZXMgeSBlc2NhbGFzIG9yaWdpbmFsZXMuDQojdF9ub3JtICogKHRfbWF4LXRfbWluKSArIHRfbWluID0gdA0KZGF0bW9kJHRlbXAgPC0gZGF5cyR0ZW1wICogKDM5LSgtOCkpIC0gOA0KI2h1bV9ub3IgPSBodW0vMTAwDQpkYXRtb2QkaHVtIDwtIGRheXMkaHVtICogMTAwDQojd2luZF9ub3IgPSB3aW5kLzY3DQpkYXRtb2Qkd2luZHNwZWVkIDwtIGRheXMkd2luZHNwZWVkICogNjcNCg0KbGlicmFyeShwZHApDQpsaWJyYXJ5KHZpcCkNCg0KI0VudHJlbmFtb3MgZWwgbW9kZWxvDQpkYXlzX3JmIDwtIHJhbmRvbUZvcmVzdChjbnQgfiAuLCBkYXRhID0gZGF0bW9kLCBpbXBvcnRhbmNlID0gVFJVRSkNCg0KcDEgPC0gcGFydGlhbChkYXlzX3JmLCBwcmVkLnZhciA9ICJkYXlzX3NpbmNlXzIwMTEiLCBwbG90ID0gVFJVRSwgcnVnID0gVFJVRSwgcGxvdC5lbmdpbmUgPSAiZ2dwbG90MiIpDQpwMiA8LSBwYXJ0aWFsKGRheXNfcmYsIHByZWQudmFyID0gInRlbXAiLCBwbG90ID0gVFJVRSwgcnVnID0gVFJVRSwgcGxvdC5lbmdpbmUgPSAiZ2dwbG90MiIpDQpwMyA8LSBwYXJ0aWFsKGRheXNfcmYsIHByZWQudmFyID0gImh1bSIsIHBsb3QgPSBUUlVFLCBydWcgPSBUUlVFLCBwbG90LmVuZ2luZSA9ICJnZ3Bsb3QyIikNCnA0IDwtIHBhcnRpYWwoZGF5c19yZiwgcHJlZC52YXIgPSAid2luZHNwZWVkIiwgcGxvdCA9IFRSVUUsIHJ1ZyA9IFRSVUUsIHBsb3QuZW5naW5lID0gImdncGxvdDIiKQ0KDQpzdWJwbG90KHAxLHAyLHAzLHA0LCBzaGFyZVggPSBGLCB0aXRsZVggPSBUUlVFKQ0KYGBgDQoNCkFuYWx5c2UgdGhlIGluZmx1ZW5jZSBvZiAqKmRheXMgc2luY2UgMjAxMSoqLCAqKnRlbXBlcmF0dXJlKiosICoqaHVtaWRpdHkqKiBhbmQgKip3aW5kIHNwZWVkKiogb24gdGhlIHByZWRpY3RlZCBiaWtlIGNvdW50cy4NCg0KQXMgdGltZSBnb2VzIG9uLCB0aGUgbW9kZWwgcHJlZGljdHMgYW4gaW5jcmVhc2UgaW4gdGhlIG51bWJlciBvZiByZW50ZWQgYmljeWNsZXMsIHdoaWNoIGlzIG5vcm1hbCBhcyB0aGUgc2VydmljZSBiZWNvbWVzIG1vcmUgd2VsbC1rbm93biBvdmVyIHRpbWUuIEZvciB3YXJtIGJ1dCBub3QgdG9vIGhvdCBjbGltYXRlcywgYSBsYXJnZSBudW1iZXIgb2YgcmVudGVkIGJpa2VzIGlzIHByZWRpY3RlZC4gWWV0LCBmcm9tIHRlbXBlcmF0dXJlcyBvdmVyIDI3IMK6QywgdGhlIG51bWJlciBvZiByZW50ZWQgYmlrZXMgZGVjcmVhc2UodG9vIG11Y2ggaGVhdCkuIEl0IGFwcGVhcnMgdGhhdCBjeWNsaXN0cyBhcmUgaW5jcmVhc2luZ2x5IGluaGliaXRlZCBmcm9tIHJlbnRpbmcgYSBiaWtlIHdoZW4gaHVtaWRpdHkgZXhjZWVkcyA2MCUuIEZpbmFsbHksIHRoZSB3aW5kaWVyIGl0IGdldHMsIHRoZSBsZXNzIHBlb3BsZSBsaWtlIHRvIHJpZGUgYSBiaWtlLiB3aGljaCBpcyBsb2dpYy4gSXQgYXBwZWFycyB0aGF0IHRoZSBtb2RlbCBwcmVkaWN0cyB0aGUgc2FtZSBmcm9tIDI1IGttL2gsIG1heWJlIGJlY2F1c2UgdGhlcmUgaXMgbGl0dGxlIHRyYWluaW5nIGRhdGEgaW4gdGhhdCByYW5nZS4NCg0KIyMjIDIuLSBCaWRpbWVuc2lvbmFsIFBhcnRpYWwgRGVwZW5kZW5jeSBQbG90Lg0KDQpHZW5lcmF0ZSBhIDJEIFBhcnRpYWwgRGVwZW5kZW5jeSBQbG90IHdpdGggaHVtaWRpdHkgYW5kIHRlbXBlcmF0dXJlIHRvIHByZWRpY3QgdGhlIG51bWJlciBvZiBiaWtlcyByZW50ZWQgZGVwZW5kaW5nIG9uIHRob3NlIHBhcmFtZXRlcnMuDQoNClNob3cgdGhlIGRlbnNpdHkgZGlzdHJpYnV0aW9uIG9mIGJvdGggaW5wdXQgZmVhdHVyZXMgd2l0aCB0aGUgMkQgcGxvdCBhcyBzaG93biBpbiB0aGUgY2xhc3Mgc2xpZGVzLg0KDQpgYGB7cn0NCmxpYnJhcnkodGljdG9jKQ0KDQoNCiNTZWxlY2Npb25hbW9zIHVuIG7Dum1lcm8gZGUgZmlsYXMgYWxlYXRvcmlhcw0Kc2FtcGxlZCA8LSBzYW1wbGVfbihkYXRtb2QsIDQwKSANCg0KI0ZpbGFzIGRlbCBhdHJpYnV0byAndGVtcCcgcXVlIHRpZW5lbiB1bmEgZXF1aXZhbGVuY2lhIGVuIGVsIGF0cmlidXRvICdodW0nDQp0ZW1waHVtIDwtIGlubmVyX2pvaW4oZGF0YS5mcmFtZShzYW1wbGVkJHRlbXApLCBkYXRhLmZyYW1lKHNhbXBsZWQkaHVtKSwgYnk9Y2hhcmFjdGVyKCkpIA0KY29sbmFtZXModGVtcGh1bSkgPC0gYygidGVtcGVyYXR1cmUiLCJodW1pZGl0eSIpDQoNCnRlbXBodW0kcHJvYiA8LSAwIA0KZm9yKGkgaW4gMTpucm93KHRlbXBodW0pKXsNCnIgPC0gZGF0bW9kDQpyW1sidGVtcCJdXSA8LSB0ZW1waHVtW1sidGVtcGVyYXR1cmUiXV1baV0NCnJbWyJodW0iXV0gPC0gdGVtcGh1bVtbImh1bWlkaXR5Il1dW2ldDQpwcmVkIDwtIHByZWRpY3QoZGF5c19yZiwgcikgDQp0ZW1waHVtW1sicHJvYiJdXVtpXSA8LSBzdW0ocHJlZCkgLyBucm93KGRhdG1vZCkgDQp9DQoNCmdncGxvdCh0ZW1waHVtLCBhZXMoeD10ZW1wZXJhdHVyZSwgeT1odW1pZGl0eSkpICsgZ2VvbV90aWxlKGFlcyhmaWxsPXByb2IsIHdpZHRoPTEwLCBoZWlnaHQ9MTApKSArIGxhYnMoeD0iVGVtcGVyYXR1cmUiLCB5PSJIdW1pZGl0eSIpICsgZ3VpZGVzKGZpbGw9Z3VpZGVfbGVnZW5kKHRpdGxlPSJOdW1iZXIgb2YgYmlrZXMiKSkgKyBnZW9tX3J1ZygpDQoNCmBgYA0KDQoqKlFVRVNUSU9OOioqDQoNCkZvciB0aGUgYW5hbHlzaXMgb2YgdGhlIHR3by1kaW1lbnNpb25hbCBQRFAgd2UgbXVzdCBjb25zaWRlciBpbiB1bmlzb24gdGhlIGRlbnNpdHkgZ3JhcGhzIG9mIGVhY2ggb2YgdGhlIGF0dHJpYnV0ZXMgKHJlcHJlc2VudGVkIG9uIHRoZWlyIHJlc3BlY3RpdmUgYXhlcykgd2l0aCB0aGUgbGVnZW5kIHdoaWNoLCBieSB0aGUgaW50ZW5zaXR5IG9mIHRoZSBjb2xvdXIsIGluZGljYXRlcyB0aGUgZXN0aW1hdGVkIHZhbHVlIG9mIHRoZSBudW1iZXIgb2YgYmljeWNsZXMgcmVudGVkLiBUaHVzLCB0aGlzIGxlZ2VuZCBzaG93cyB0aGF0IHRoZSBhcmVhcyB3aXRoIGEgbGlnaHRlciBibHVlIGNvbG91ciBpbmRpY2F0ZSBhIGhpZ2hlciB2YWx1ZSBvZiB0aGUgcHJlZGljdGVkIHJlc3BvbnNlIHZhcmlhYmxlLCBhbmQgYWx0ZXJuYXRpdmVseSB0aGUgZGFya2VyIHRvbmVzIGNvcnJlc3BvbmQgdG8gbG93ZXIgZXN0aW1hdGVkIHZhbHVlcyBmb3IgdGhlIG51bWJlciBvZiBiaWN5Y2xlcy4NCg0KV2UgZm9jdXMgb3VyIGV4cGxhbmF0aW9uIG9uIHRoZSByYW5nZXMgb2YgdmFsdWVzIG9mIG91ciBhdHRyaWJ1dGVzIGZvciB3aGljaCB3ZSBjYW4gZHJhdyBnZW5lcmFsIGludGVycHJldGF0aW9ucyB3aXRoIHJlc3BlY3QgdG8gdGhlIHByZWRpY3RpdmUgbW9kZWwuIEluIG91ciBjYXNlIGZvciB0ZW1wZXJhdHVyZSAoZnJvbSA0wrpDIHRvIDI4wrpDKSBhbmQgZm9yIGh1bWlkaXR5IChmcm9tIDQ0JSB0byA4MSUpIHdlIG9ic2VydmUgbmluZSBjbGVhcmx5IGRpZmZlcmVudGlhdGVkIHpvbmVzIGluIHRlcm1zIG9mIHRvbmUuDQoNCkluIHRoZSBmaXJzdCB6b25lIChUZW1wZXJhdHVyZTogNC04wrpDLCBIdW1pZGl0eTogNzUtODElKSB3ZSBvYnNlcnZlIHRoYXQgaXQgaXMgd2hlcmUgdGhlIG51bWJlciBvZiByZW50ZWQgYmljeWNsZXMgaXMgdGhlIGxvd2VzdCAoYmV0d2VlbiAzMDAwIGFuZCAzNTAwIHVuaXRzKSBqdXN0IHdoZXJlIGxvdyB0ZW1wZXJhdHVyZSBhbmQgaGlnaCBodW1pZGl0eSBjb25kaXRpb25zIG1lZXQuDQoNCkluIHRoZSBsYXN0IHpvbmUgKHRlbXBlcmF0dXJlOiAxNy0yOMK6QywgaHVtaWRpdHk6IDQ0LTYxJSkgZGVsaW1pdGVkIGJ5IGhpZ2ggdGVtcGVyYXR1cmUgYW5kIGxvdyBodW1pZGl0eSBjb25kaXRpb25zLCB0aGUgaGlnaGVzdCBudW1iZXIgb2YgcmVudGVkIGJpY3ljbGVzIGlzIHByZWRpY3RlZCAoYXBwcm94aW1hdGVseSA1MDAwIHVuaXRzKS4NCg0KQm90aCBib3VuZGFyeSBzaXR1YXRpb25zIGNvcnJlc3BvbmQgdG8gdGhlIGRpcmVjdCByZWxhdGlvbnNoaXAgYmV0d2VlbiBvdXIgcmVzcG9uc2UgdmFyaWFibGUgd2l0aCB0aGUgdmFyaWFibGUgJ3RlbXAnIGFzIHdlbGwgYXMgdGhlIGludmVyc2UgcmVsYXRpb25zaGlwIHdpdGggdGhlIHZhcmlhYmxlICdodW0nLg0KDQojIyMgKiozLi0gUERQIHRvIGV4cGxhaW4gdGhlIHByaWNlIG9mIGEgaG91c2UuKioNCg0KQXBwbHkgdGhlIHByZXZpb3VzIGNvbmNlcHRzIHRvIHByZWRpY3QgdGhlICoqcHJpY2UqKiBvZiBhIGhvdXNlIGZyb20gdGhlIGRhdGFiYXNlICoqa2NfaG91c2VfZGF0YS5jc3YqKi4gSW4gdGhpcyBjYXNlLCB1c2UgYWdhaW4gYSByYW5kb20gZm9yZXN0IGFwcHJveGltYXRpb24gZm9yIHRoZSBwcmVkaWN0aW9uIGJhc2VkIG9uIHRoZSBmZWF0dXJlcyAqKmJlZHJvb21zKiosICoqYmF0aHJvb21zKiosICoqc3FmdF9saXZpbmcqKiwgKipzcWZ0X2xvdCoqLCAqKmZsb29ycyoqIGFuZCAqKnlyX2J1aWx0KiouDQoNClVzZSB0aGUgcGFydGlhbCBkZXBlbmRlbmNlIHBsb3QgdG8gdmlzdWFsaXplIHRoZSByZWxhdGlvbnNoaXBzIHRoZSBtb2RlbCBsZWFybmVkLg0KDQpgYGB7cn0NCiNTZWxlY2Npw7NuIGRlIGZpbGFzIGFsZWF0b3JpYXMNCnNhbXBsZV9ob3VzZSA8LSBzYW1wbGVfZnJhYyhob3VzZSwgMC4yKQ0Kc2FtcGxlX2hvdXNlIDwtIHNlbGVjdChzYW1wbGVfaG91c2UsIGJlZHJvb21zLCBiYXRocm9vbXMsIHNxZnRfbGl2aW5nLCBzcWZ0X2xvdCwgZmxvb3JzLCB5cl9idWlsdCwgcHJpY2UpDQoNCiNFbnRyZW5hbW9zIGVsIG1vZGVsbw0KaG91c2VfcmYgPC0gcmFuZG9tRm9yZXN0KHByaWNlIH4gLiwgZGF0YSA9IHNhbXBsZV9ob3VzZSwgaW1wb3J0YW5jZSA9IFRSVUUpDQoNCnAxIDwtIHBhcnRpYWwoaG91c2VfcmYsIHByZWQudmFyID0gImJlZHJvb21zIiwgcGxvdCA9IFRSVUUsIHJ1ZyA9IFRSVUUsIHBsb3QuZW5naW5lID0gImdncGxvdDIiKQ0KcDIgPC0gcGFydGlhbChob3VzZV9yZiwgcHJlZC52YXIgPSAiYmF0aHJvb21zIiwgcGxvdCA9IFRSVUUsIHJ1ZyA9IFRSVUUsIHBsb3QuZW5naW5lID0gImdncGxvdDIiKQ0KcDMgPC0gcGFydGlhbChob3VzZV9yZiwgcHJlZC52YXIgPSAic3FmdF9saXZpbmciLCBwbG90ID0gVFJVRSwgcnVnID0gVFJVRSwgcGxvdC5lbmdpbmUgPSAiZ2dwbG90MiIpDQpwNCA8LSBwYXJ0aWFsKGhvdXNlX3JmLCBwcmVkLnZhciA9ICJmbG9vcnMiLCBwbG90ID0gVFJVRSwgcnVnID0gVFJVRSwgcGxvdC5lbmdpbmUgPSAiZ2dwbG90MiIpDQoNCnN1YnBsb3QocDEscDIscDMscDQsIHNoYXJlWCA9IEZBTFNFLCB0aXRsZVggPSBUUlVFKQ0KDQpgYGANCg0KKipRVUVTVElPTjoqKg0KDQoqKipBbmFseXNlIHRoZSBpbmZsdWVuY2Ugb2YgYmVkcm9vbXMsIGJhdGhyb29tcywgc3FmdF9saXZpbmcgYW5kIGZsb29ycyBvbiB0aGUgcHJlZGljdGVkIHByaWNlLioqKg0KDQpUaGUgZ2VuZXJhbCBpbnRlcnByZXRhdGlvbiBvZiB0aGUgJ3Bvc3QgaG9jJyBQRFAgbW9kZWwgZm9yIHRoZSBhdHRyaWJ1dGUgKionYmVkcm9vbXMnKiogd2l0aCByZXNwZWN0IHRvIHRoZSBwcmVkaWN0ZWQgbW9kZWwgYWNjb3JkaW5nIHRvIGl0cyBkZW5zaXR5IGdyYXBoIGFsbG93cyB1cyB0byBkcmF3IGNvbmNsdXNpb25zIGluIHRoZSByYW5nZSBvZiAxIHRvIDQgYmVkcm9vbXMuIFdlIG9ic2VydmUgdHdvIGRpZmZlcmVudCB0cmVuZHM6IGFuIGluY3JlYXNpbmcgdHJlbmQsIGkuZS4gYSByaXNlIGluIHByaWNlIGZyb20gMSB0byAyIGJlZHJvb21zIChmcm9tIDU3MjAwMCB0byA1ODEwMDAkKTsgd2hpbGUgYSBkZWNyZWFzZSBpbiBwcmljZSBmcm9tIDIgdG8gNCBiZWRyb29tcyAoZnJvbSA1ODEwMDAgdG8gNTQyMDAwJCkuIFdlIGNhbiB1bmRlcnN0YW5kIHRoYXQgdGhlIG1vc3QgcmVxdWVzdGVkIG51bWJlciBvZiByb29tcyBwZXIgaG91c2UsIGFuZCB0aGVyZWZvcmUgdGhlIG1vc3QgZXhwZW5zaXZlLCBhcmUgdGhvc2Ugd2l0aCAyIHJvb21zLiBJbiB0aGUgcmFuZ2UgYWJvdmUgdGhlc2UsIGFzIGl0IGlzIG5vdCBhIHVzdWFsIG51bWJlciBvZiByb29tcywgd2UgaW50ZXJwcmV0IHRoaXMgYXMgdGhlIHJlYXNvbiB3aHkgdGhlIHByZWRpY3RpdmUgbW9kZWwganVzdGlmaWVzIHRoZSB0cmVuZCBkZXNjcmliZWQgYWJvdmUuDQoNCkFzIGZvciB0aGUgdmFyaWFibGUgKionYmF0aHJvb21zJyoqIHdlIGNhbiBvYnNlcnZlIGEgcG9zaXRpdmUgY29ycmVsYXRpb24gYmV0d2VlbiBpdCBhbmQgdGhlIGVzdGltYXRlZCBwcmljZSBvZiB0aGUgaG91c2UsIHNpbmNlIG5vcm1hbGx5IGEgaG91c2Ugd2l0aCBtb3JlIGJhdGhyb29tcyBpcyBjb25zaWRlcmVkIGEgbW9yZSBsdXh1cmlvdXMgaG91c2UgYW5kIHRoZXJlZm9yZSBtb3JlIGV4cGVuc2l2ZSwgaXQgaXMgd29ydGggbm90aW5nIHRoZSBjbGVhciBpbmNyZWFzZSBpbiBwcmljZSBpZiB3ZSBnbyBmcm9tIDMgdG8gNCBiYXRocm9vbXMuIFdlIGNhbiBvbmx5IGRyYXcgdmFsaWQgY29uY2x1c2lvbnMgaWYgd2UgdGFrZSBpbnRvIGFjY291bnQgdGhvc2UgZHdlbGxpbmdzIHdpdGggMCB0byA0IGJhdGhyb29tcywgYXMgd2UgZG8gbm90IGhhdmUgZW5vdWdoIGRhdGEgcmVjb3JkZWQgZm9yIGR3ZWxsaW5ncyB3aXRoIGEgaGlnaGVyIG51bWJlciBvZiBiYXRocm9vbXMuDQoNCkFzIGZvciB0aGUgdmFyaWFibGUgKionc3FmdF9saXZpbmcnKiosIHRoZSByZWxhdGlvbnNoaXAgaXMgc2ltaWxhciB0byB0aGUgY2FzZSBvZiBiYXRocm9vbXMsIGEgbGFyZ2VyIGR3ZWxsaW5nIHdpbGwgb2J2aW91c2x5IGJlIG1vcmUgZXhwZW5zaXZlIGFzIGl0IG9jY3VwaWVzIG1vcmUgbGFuZC4gV2UgaGF2ZSBkYXRhIHVwIHRvIGFib3V0IDUwMDAgc3F1YXJlIG1ldHJlcywgZm9yIHRob3NlIGR3ZWxsaW5ncyB3aG9zZSBzaXplIGV4Y2VlZHMgdGhpcywgd2UgZG8gbm90IGhhdmUgZW5vdWdoIGRhdGEgcmVjb3JkZWQgYW5kIHRoZXJlZm9yZSBjYW5ub3QgZHJhdyB2YWxpZCBjb25jbHVzaW9ucy4NCg0KRm9yIHRoZSBsYXN0IFBEUCBncmFwaCwgaW4gcmVsYXRpb24gdG8gdGhlIGFuYWx5c2lzIG9mIHRoZSB2YXJpYWJsZSAqKidmbG9vcnMnKiogYmVpbmcgdGhlIGluY3JlYXNpbmcgdHJlbmQgd2Ugb2JzZXJ2ZSBhIG5vdGFibGUgZGlmZmVyZW5jZSBpbiB0aGUgcHJpY2Ugb2YgdGhlIGhvdXNlIHdoZW4gZ29pbmcgZnJvbSBvbmUgZmxvb3IgdG8gdHdvLCBhbmQgYWJvdmUgYWxsLCBpZiB0aGVyZSBpcyBhIHRoaXJkIGZsb29yIGluIHRoZSBob3VzZS4gQ29uc2lkZXJpbmcgdGhlIHNtYWxsIGRpZmZlcmVuY2UgYmV0d2VlbiBvbmUgZmxvb3IgYW5kIHR3byBmbG9vcnMgKGZyb20gNTQwMDAwIHRvIDU2MDAwMFwkKSwgdGFraW5nIGludG8gYWNjb3VudCB0aGUgZGVtb2dyYXBoaWMgY2hhcmFjdGVyaXN0aWNzIG1lbnRpb25lZCBhYm92ZSBhbmQgY29uc2lkZXJpbmcgdGhlIGNsaW1hdG9sb2dpY2FsIGNoYXJhY3RlcmlzdGljcyBvZiB0aGUgYXJlYSAoc2hvcnQsIGhvdCBhbmQgZHJ5IHN1bW1lcnMgYW5kIGNvbGQgYW5kIHdldCB3aW50ZXJzKSB0aGUgY29uc3RydWN0aW9uIGVzdGltYXRlZCBieSB0aGUgdHdvLWZsb29yIG1vZGVsIGlzIHVzdWFsbHkgdGhlIG1vc3QgZGVtYW5kZWQsIHBsYWNpbmcgdGhlIGxpdmluZyBhcmVhcyBhbmQga2l0Y2hlbiBvbiBvbmUgZmxvb3IgYW5kIHRoZSBiZWRyb29tcyBvbiB0aGUgdXBwZXIgZmxvb3IuIE9mIGNvdXJzZSwgYSB0aGlyZCBmbG9vciBpcyBhbHJlYWR5IGEgYmlnIGJ1ZGdldCBkZXZpYXRpb24gYW5kIGlzIGNvbnNpZGVyZWQgdG8gYmUgYW4gZXhjZXB0aW9uYWwgaG91c2UuDQo=